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-- The MAILING DATE of this communication appears on the cover sheet with the correspondence address -- 
Period for Reply 

A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) FROM 
THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1 .136(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

• If the period for reply specified above is less than thirty (30) days, a reply within the statutory minimum of thirty (30) days will be considered timely. 

- if NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 133). 

- Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1.704(b). 

Status 

1)13 Responsive to communication(s) filed on 20 June 2002 . 
2a)D This action is FINAL. 2b)S This action is non-final. 

3) D Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 11, 453 O.G. 213. 
Disposition of Claims 

4) H Claim(s) 1-22 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) D Claim(s) is/are allowed. 

6) H Claim(s) 1-22 is/are rejected. 

7) D Claim(s) is/are objected to. 

&*)□ Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9)D The specification is objected to by the Examiner. 

10) D The drawing(s) filed on J is/are: a)D accepted or b)D objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1.85(a). 

1 1) D The proposed drawing correction filed on is: a)D approved b)D disapproved by the Examiner. 

If approved, corrected' drawings are required in reply to this Office action. 

12) D The oath or declaration is objected to by the Examiner. 
Priority under 35 U.S.C. §§119 and 120 

13) D Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 119(a)-(d) or (f). 

aO All b)D Some*c)D None of: 

1 .□ Certified copies of the priority documents have been received. 

2.Q Certified copies of the priority documents have been received in Application No. . 



3.D Copies of the certified copies of the priority documents have been received in this National Stage 
application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 

14) D Acknowledgment is made of a claim for domestic priority under 35 U.S.C. § 119(e) (to a provisional application). 

a) □ The translation of the foreign language provisional application has been received. 

15) D Acknowledgment is made of a claim for domestic priority under 35 U.S.C. §§ 120 and/or 121. 

Attachment(s) 



1 ) |23 Notice of References Cited (PTO-892) 

2) Notice of Draftsperson's Patent Drawing Review (PTO-948) 

3) CH Information Disclosure Statement(s) (PTO-1449) Paper No(s)_ 



4) EH Interview Summary (PTO-41 3) Paper No(s). 

5) □ Notice of Informal Patent Application (PTO-152) 

6) D Other: 



U.S. Patent and Trademark Office 
PTO-326 (Rev. 04-01) 



Office Action Summary 
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DETAILED ACTION 



Response to Arguments 

1 . Applicant's arguments with respect to claims 1 -22 have been considered but are moot in 
view of the new ground(s) of rejection. 



2. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all obviousness 
rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

3. Claims 1-22 are rejected under 35 U.S.C. 103(a) as being unpatentable over Szeliski et 
al., U.S. Pat. No. 6,157,747 in view of Eppler, U.S. Pat. No. 6,084,989. 

Considering claim 1, Szeliski discloses the following claimed subject matter, note; 

a) the claimed method of extracting a sequence of video frames is met by image frames Io-Ik 

inputted into the step 110, FIG. 1 ; (see also FIGS. 3 and 6). 

d) the claimed method of aligning the interpolated video frames is met by step 120, FIG.l; 

e) the claimed method of creating a single image from the aligned video frames is met by output 
image, FIG.l, which constructs an image mosaic; 



Claim Rejections - 35 USC § 103 
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Except for; 

b) the claimed method of sampling each of the video frames; 

c) the claimed method of interpolating the up sampled video frames; 

Regarding b), Szeliski et al. discloses 3 -dimensional image rotation method and apparatus 
for producing image mosaics. Szeliski doesn't specifically disclose^'up sampling". However, 
Szeliski et al. discloses A "sampling" and "resampling" of the frames Io-Ik in, for example, step 
860, FIG.8; and, resamples the output version of Ik (x) in step 2080 (FIG.20). 

Eppler discloses a system and method for automatically determining the position of 
landmarks in digitized images derived from a satellite based imaging system. Eppler discloses a 
landmark positioning system 20 comprising the Image Restoration (upsampler 7x4) 24 that up 
samples the image patch. 

Therefore, it would have been obvious to the skilled in the art at the time the invention 
was made to provide the up sampler 24 of Eppler and modify the system of Szeliski in order to 
provide the latter with an upsampling capability. 

Regarding c), Szeliski discloses a method of interpolation (step 2060, FIG.20) and 
modification of alignment parameters (step 1050, FIG. 13) and data interpolation (step 2640, Fig. 
20). The system of Szeliski performs the claimed functions such as interpolation and sampling in 
reverse order, that is, alignment comes first, then sampling and interpolation, etc. However, it 
would have been obvious to the skilled in the art at the time the invention was made that the order 
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of performing the functions such as interpolation and sampling would merely be an obvious 
engineering choice. 



Considering claim 2, the claimed method wherein the sequence of video frames are low resolution 
images is met by Fig. 10 which illustrates a coarse-to-fine resolution hierarchy employed in one 
implementation, i.e., starting with low resolution. 

Considering claim 3, the claimed method wherein said up sample step is by a factor of 4 . 
Regarding claim 3, see rejection of claim 1 (b). 

Considering claim 4, the claimed method wherein said align step comprises aligning the video 
images in an x direction and a y direction in a center portion of interest in each video frame is 
inherent, because alignment of images implies in all directions. (See also FIG.3) 

Considering claim 5, the claimed comprising extracting the sequence of video frames at 30" 
frames/sec is inherent, because the rate of 30 frames/sec is standard in video/television systems. 

Considering claim 6, the claimed wherein the sequence of video frames includes 5 video frames is 
met by initial frames Io-Ik; (FIG.l) 
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Considering claim 7, the claimed method of correlating the up sampled video images is met by the 
"Patch-based algorithm" step 120, FIG.L (See also rejection of claim 1 (b)); 

Considering claim 8, the claimed rising averaging a pixel intensity from each of the up sampled 
video frames is met by the disclosure that "The blending of the corresponding pixel value employs 
a weighted average according to" the formula given on col. 30, lines 1-3. 

Considering claim 9, the claimed comprising compensating for platform movement and rotation 
zoom is met by the disclosure that "Based on this rotational panoramic representation, block 
adjustment (global alignment) and degho sting (local alignment) methods disclosed herein 
significantly improve the quality of image mosaics, thereby enabling the construction of mosaics 
from images taken by hand-held cameras." (Col. 32, lines 12-18) 

Considering claim 10, the claimed aligning each the extracted sequence of video frames before 
said up sample step; 

Regarding claim 10, see rejection of claim l(c ). 

Considering claim 11, identifying commonality from one individual frame to the next and 
overlapping the individual frames and displaying an image representing a continuous area is met 
by FIG. 3, which illustrates overlapping individual frames and displaying a panoramic image 
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Considering claim 12, extracting the sequence of video frames at 30 frames/sec is inherent, 
because the rate of 30 frames/sec is standard in television systems. 

Considering claim 13, the claimed wherein the sequence of video frames includes 5 video frames. 
Regarding claim 13, see rejection of claim 6. 

Considering claim 14, the claimed correlating the up sampled video images is met by the "Patch- 
based algorithm" step 120, FIG.l. (See also rejection of claim 1 (b)); 

Considering claim 15, the claimed averaging a pixel intensity from each of the up sampled video 
frames. 

Regarding claim 15, see rejection of claim 8. 

Considering claim 16, The claimed comprising compensating for platform movement and rotation 
zoom. 

Regarding claim 16, see rejection of claim 9. 

Considering claim 17, the claimed method of aligning each [of] the extracted sequence of video 
frames before said up sample step. 
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Regarding claim 17, see rejection of claim 1 (c ). 

Considering claim 18, identifying commonality from one individual frame to the next and 
overlapping the individual frames and displaying an image representing a continuous area. 
Regarding claim 18, see rejection of claim 11. 

Considering claim 19, see rejection of claim 5. 

Considering claim 20, see rejection of claim 1. 

Considering claim 21, 

a) the claimed at least one sequence of machine executable instructions is met by the Application 
Programs 36 in System Memory 22 of the PC 20, FIG. 2A. 

b) the claimed a medium bearing the executable instructions in machine form, wherein execution 
of the instructions by one or more processor causes the one or more processor to extract a 
sequence of video frames, up sample each of the video frames; interpolate the up sampled video 
frames; align the interpolated video frames; and create a single image from the aligned video 
frames; 

Regarding b), see rejection of claim 1. (See also Fig.2A) 
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Considering claim 22, Szeliski discloses the following claimed subject matter, note; 

a) the claimed processor is met by Processing unit 21, FIG. 2 A; 

b) the claimed memory coupled to said processor is met by system memory which includes ROM 
24 and RAM 25, FIG. 2A; 

c) the claimed memory having stored therein sequences of instructions when executed by said 
processor, causes said processor to perform the steps is met by the Applications Programs, 36 
FIG. 2A. 

d) the claimed steps of extract a sequence of video frames; up sample each of the video frames; 
interpolate the up sampled video frames; align the interpolated video frames; and create a single 
image from the aligned video frames; 

Regarding d), see rejection of claim 1. 

Conclusion 

4. Any inquiry concerning this communication or earlier 
communications from the examiner should be directed to Paulos 
Natnael whose telephone number is (703) 305-0019. The examiner 
can normally be reached on Monday through Friday from 6:30 a.m. 
to 3:00 p.m. 



* 
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If attempts to reach the examiner by telephone are 
unsuccessful, the examiner's supervisor, John Miller, can be 
reached on (703)305-4795. 

Any inquiry of a general nature or relating to the status of 
this application or proceeding should be directed to the Group 
receptionist whose telephone number is (703)305-3900. 
Any response to this action should be mailed to: 
Commissioner of Patents and Trademarks 
Washington, D.C. 20231 

or faxed to: 

(703) 872-9314, (for formal communications intended for 
entry) 



(703)872-9314 (for informal or draft communications, 
please label "PROPOSED" OR " DRAFT" ) . 
Hand-delivered responses should be brought to Crystal Park 
II, 2121 Crystal Drive, Arlington, V.A. Sixth Floor 
(Receptionist) . 
Paulos M. Natnael A \ 



or: 





September 4, 2 0 02 



^ JOHN MILLER 
SUPERVISORY PATENT EXAMINER 
TECHNOLOGY CENTER 2600 



